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ABSTRACT 


Output  probabilities  of  finite- state -machine  elements  are  studied  when 
the  input  of  the  machine  elements  is  a  random  process.  Delay  element, 
modulo  m  adder  and  modulo  m  multiplier  are  considered  as  the  machine 
elements.  A  new  approach  to  the  problem  is  made  by  defining  the  trans¬ 
formation  matrices  T  and  U  for  a  modulo  m  adder  and  a  modulo  m 
—  — a 

multiplier  respectively.  By  using  these  matrices,  the  output  probabilities 
of  the  machine  and  the  stochastic  matrix  of  the  output  are  given  in 
explicit  expressions,  assuming  that  the  input  process  is  a  finite- state, 
time -invariant  Markov  process.  A  few  examples  of  the  combin?.torial 
circuit  of  these  elements  are  also  studied. 
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I.  INTRODUCTION 


A  few  probabilistic  input-output  relations  of  finite- state- machine 
elements  are  derived  for  a  random  input.  We  defined  the  transforma¬ 
tion  matrices  T  and  U  ^  in  order  to  express  the  output  probabilities 
in  explicit  relations,  which  are  the  matrices  with  entries  of  I's  and 
O's.  A  few  properties  of  these  matrices  are  studied.  T  is  the  trans¬ 
formation  matrix  for  the  modulo  m  adder  and  U  is  that  for  the  mo- 

—  a 

dulo  m  multiplier.  If  we  restrict  ourselves  to  a  Galois  field,  or  in 
other  words,  if  m  is  a  prime,  we  can  obtain  the  consistent  expres¬ 
sions  for  tlie  probabilitistic  relations  both  for  the  modulo  m  adder 
and  for  the  modulo  m  multiplier. 

Conditional  probability  of  the  two  inputs  of  the  machine  is  consi¬ 
dered.  In  particular,  modulo  2  multiplier  is  an  input-dependent  ran¬ 
dom  switch.  Combinatorial  circuits  using  these  modular  circuit  ele¬ 
ments  are  analyzed  in  terms  of  the  conditional  probability  and  the 
absoute  probability  of  the  inputs.  The  stochastic  matrix  of  the  output 
of  a  few  combinatorial  circuits  is  also  given  in  terms  of  the  input 
stochastic  matrix  and  the  transition  matrices  we  defined,  when  the 
input  is  a  finite- state,  time- invariant  Markov  process. 

II.  BASIC  ELEMENTS  OF  MODULAR  CIRCUITS 

We  will  consider  a  symbol  x  which  belongs  to. a  set  of  symbols 
S  with  m  distinct  component  symbols  such  that 

S=  {0,1,  2,...,m-l}  (2.1) 

We  define  the  absolute  probability  of  a  symbol  x  at  time  t  as 

P^(x=0) 

Pt  (x-1) 

Pt(x)=  .  (2.2) 

(x=m-l) 
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where 


P^(x)  =  Pr(x=x  at  time  t). 

Suffix  t  will  be  omitted  elsewhere  through  this  paper  unless  it  is  neces¬ 
sary.  Note  that  Pj(x)  is  not  the  function  of  x  but  rather  of  t  as  is  seen 
in  the  definition  (2.  2).  However,  we  use  this  notation  for  convenience. 

A-  Delay  Element' 

If  we  denote  an  input  and  an  output  at  time  t  by  x(t)  and  z(t), respec¬ 
tively,  and  the  delay  operator  by  D,  then 


z(t)  =  D  x(t)  =  x(t-l) 


(2.  3) 


Therefore,  we  have 


P^(z)=  Pt-i(x) 


(2.4) 


where  x,  zc  S. 


B.  Modulo  m  Adder 


We  define  an  operator  ®  of  addition  modulo  m  such  that 

z=a  (J)  X  =  a+x  mod  m  (2.  5) 


where  a,  x,  z  c  S. 
Example 


aV 

0  12  3  4 

0 

0  12  3  4 

1 

1  2  3  4  0 

2 

2  3  4  0  1 

3 

3  4  0  1  2 

4 

4  0  12  3 

z  =  a(J)x  mod  5 
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As  is  seen  in  the  above  example,  the  vector  z  is  obtained  by 
shifting  the  vector  x  by  an  amount  equal  to  "a,",  cyclically  to  the 
descending  direction  of  x.  Therefore,  by  defining  a  transformation 
T  such  that 


0 

0 

0  .  .  . 

0 

1 

1 

0 

0  ... 

0 

0 

0 

1 

0  .  .  . 

0 

0 

0 

0 

1  ... 

0 

0 

0 

0 

0  . 

1 

0 

m  X  m 


(2.6) 


we  have  the  following  relation; 


P(z|  a)  =  T^P(x),  X,  z,a«S  (2.7) 

where  P(z  |  a)  is  P(z)  when  "a"  is  known. 

It  is  easy  to  see  that  T  is  a  transformation  which  gives  the 
one-to-one  correspondence  of  x  and  z,  because  T  has  only  one  entry 
of  1  in  each  row  and  in  each  column  and  0  elsewhere  for  any  a€  S. 
Now,  we  have  the  basic  relation 


T  =  T 


(2.  8) 


T  is  periodic  with  period  m  and 


t'^=T*^  =  I=  identity  matrix 


Also  we  can  verify 


m-1 

I 

k=0 


ct 

T  is  an  orthogonal  matrix  for  all  a  t  S,  because 
T^'  T  ^  =  I  Va-€  S 


1  1  1  .  .  .  .  1 

1  1  1  .  . .  .  1 
1  1  1  .  .  .  .  1 

1  1  1  ,  .  .  .  1 


(2.9) 


(2.  10) 


Therefore,  T  yields  an  orthogonal  transformation  of  P(x). 
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More  in  general,  if  we  consider  a  modulo  m  adder  with  one  random 
input  and  k  constant  inputs  Cp  C2, .  .  .  .  ,  Cj^  as  depicted  in  the  following 
figure,  then  the  left  side  figure  is  equivalent  to  that  of  the  right  side, 
since  the  associative  law  holds  for  the  operator  0  . 
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or  equivalently 

m-1 

P(z)  P(x)T^P(y|x)  (2.13) 

x=0 


In  another  expression,  since  a  component  of  P(z)  is  the  skew  sum 
of  the  joint  probability,  matrix  of  x  and  y,  we  can  calculate  it 
according  to  the  following  f^rmulct,_ _ _ _ 


p(0,l)  .  ^(O.'^-l) 

^1-rO)  p{l,l)  ^  p(l,  m-J)' 


E^m-4,'0)  p(m-l,  Ij,  .<  ^p(m-l,  m-1) 


P(z=0) 

P(z=l) 

P(z=m-1)' 


where  p(x,  y)  is  the  joint  probability  of  x  and  y. 

Now,  we  define  a  transition  matrix  G  such  that 

G  =  [  P(x|0),  T  P(xl  1),  _T^  P(x|  2),  .  .  .  P(xj  m-1)  ] 


(2.14) 


P(OlO)  P(m-l]l)  -  P(l|m-1) 


P(1  0)  P(0|1) 

P(2  0)  P(l|l) 


...  P(2 
....  P(3 


m-1) 

m-1) 


P(m-l|0)  P(m-2|l)  -  P(0|m-1) 


(2.15) 


Then,  using  this  matrix  we  can  rewrite  eq.  (2.  13)  in  an  explicit 
formula; 


P(z)  =  G  P(y) 


(2.16) 


G  '  is  an  m  by  m  matrix  and  is  a  stochastic  matrix,  which 
satisfies 


m-1 


m-1 


=Z['Slir‘ 

j=0 


(2.17) 


ij 


i=0 
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This  can  easily  be  proved  recalling  that  is  a  transformation  with 
one-to-one  correspondence  and  that 


m-1 

^  P(i|  j)  =  1 

i=0 

If  X  and  y  are  statistically  independent,  then  P(x|  y)  =  P(x)  and 
G=[^(x),  T  P(x),  P(x) . P(x)] 


P(0) 

P(m-l) 

P(m-2)  ... 

P(l) 

P(l) 

P(0) 

P(m-1 )  ... 

P(2) 

_P(2) 

P(l) 

P(0) 

P(3) 

P(m-l) 

P(m-2) 

P(m-3)  - 

’P(O) 

From  eq.  (2.  12)  and.  knowing  that  JP(x|  y)  =  ^(x),  we  have 


m-1 


G  =  ^  P(x)2"‘ 
x=0 


Also  we  see  that 


(2-19) 


P(z) 


X  =  z-y  mod  m)  P(y) 


y 


(2.  20) 


This  implies  that  P(z)  is  given  by  the  composition  modulo  m  of 
P(x)  and  P(y). 

Furthermore,  if 

P(x|  y)  =  P(x)  -  lira  ,  xe  S 


then 


m-1 


x=  0 


1  1  1  .  .  .  1 

1  1  1  .  .  .  1 

1  1  1  .  .  .  1 

1  1  1  .  .  .  1 


(2.21) 
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Therefore,  P(z)  is  equal  to  l/m  for  all  ze  S  regardless  of  P(y),  i.  e. 


P(z)  =  — 

—  m 


(2.  22) 


This  is  also  true  for  any  P(x)  when  P(y)  =  l/m  for  all  y  €  S. 


C.  Modulo  m  Multiplier 

First,  we  will  consider  a  constant  multiplier  defined  by 


z  =  aox  =  ax  mod  m  (2.  23) 

where  a  is  a  constant,  x  is  a  random  input  and  z  is  an  output  of 
the  multiplier  as  shown  below. 


As  we  have  done  for  modulo  m  adder,  we  also  define  an  m  by 
m  transformation  matrix  U  such  that 


U 


a. . 
ij 


(2.24) 


U  ,  is  ij-th  element  of  the  matrix  U  .  A  few  examples  of  U  are 
shoi^n  in  the  appendix. 

If  X  =  j  is  transformed  by  U^,  then  from  eq.  (2.  24)  z  is  given 
by  z  =  i  =  aoj.  Therefore,  is  a  transformation  which  gives  the 
change  of  symbols  from  x  to  z  according  to  z  =  a  ox.  The  absolute 
probability  vector  of  z  for  a  given  constant  "a"  is  now  given  by 
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(2.25) 


P(z  I  a)  =  P(x) 

Before  going  into  the  probabilistic  relations  of  x,  z  and  a, 
we  will  study  more  about  U^. 

i)  U  has  exactly  one  entry  of  1  in  each  column  and  0  entries 
"Ql 

elsewhere. 

Proof.  Each  of  m  input  symbols  has  to  make  a  deterministic  transi¬ 
tion  to  a  certain  output  symbol  in  S.  If  there  were  two  or  more 
entries  of  I's  in  a  column,  then  there  would  be  two  or  more  outputs 
corresponding  to  that  input.  This  contradicts  the  definition 
of  deterministic  machine  elements.  If  there  were  no  entry  of  1  in 
a  column,  then  there  would  be  no  output  corresponding  to  the  input. 
This  contradicts  the  fact  that  the  output  symbol  should  be 
either  one  of  the  symbols  in  S.  Therefore,  there  is  exactly  one 
entry  of  1  in  each  column  of  U  and  0  entries  elsewhere,  Q.  E.  D, 

2 

ii)  Consequently,  U  has  exactly  m  entries  of  I's  and  m  -m 

—3. 

entries  of  O's. 

Now,  we  will  prove  the  following  theorem: 

Theorem  1.  If  a  /  0,  then  has  exactly  one  entry  of  1  in  each  column 
and  in  each  row,  if  and  only  if 

GCD(a,  m)  =  1,  a  e  S,  a^O  (2.26) 

where  GCD(a,m)  denotes  the  Greatest  Common  Divisor  of  a  and  m. 
Proof.  From  the  definition  of  U  by  eq.  (2.  24), 

■  ■  ■  ■  '  -  ■  ct 

i  =  aoj,  jc  S,  a  e  S,  a  ^  0 

then  there  exists  a  certain  positive  integer  k  such  that 
i=aoj  =  aj  -  km,  i  e  S 

Therefore,  GCD(a,  m)  divides  (aj-km)  and  hence,  i;  i.  e.  , 


-8- 


-  =  aoj  mod -  (2.Z7) 

GCD(a,  m)  GCD(a,  m) 

Now,  m/GCD(a,  m)  is  a  prime.  Recalling  the  theorem  of  Galois 
field,  i.  e.  ,  for  r,  x,  sc  S, 


J  Vs  V^  3  X  [  X  o  s 

[s/O 

we  have 


r  mod 


is  prime} 


(2.  28) 


VW 


^  GCD(a,  m) 


3  j[ 


GCD(a,  m) 


=  a  0  j  mod 


_ m _ 

GCD(a,  m) 


^  'GCEf(^,  m)  ^  j*  <2- 

The  number  of  I's  in  U  is  exactly  m,  and  from  the  property 

(i)  there  is  exactly  one  "1"  associated  with  j  for  all  jc  S.  Therefore, 
i  can  take  any  integer  in  S,  if  and  only  if  GCD(a,  m)  =  1.  From  the 
property  (i),  there  is  exactly  one  entry  of  1  in  j-th  column  for  all  j 
in  S.  Therefore,  there  is  exactly  one  entry  of  1  in  the  i-th  row  for 
all  is  S,  if  and  only  if  GCD(a,  m)  =  1.  Q.  E.  D. 


Consequence.  There  is  one-to-one  correspondence  between  input 
symbols  and  output  symbols,  if  and  only  if  GCD(a,  m)  =  1. 


=  U,  U  =  U  „  , 

31 '  D  Id  — —  cL  ■■■■  31  ®  D 


Theorem  2.  For  all  a,  be  S, 

(2.  30) 

Proof.  Let  us  take  a,  b,  i,  jc  S,  then  by  the  definition  of  U^, 

1,  i  =  a  o  k 


U 


^ik 


U 


[o,  i  ^  a  o  k 

1,  k=  b  o  j 


0,  k^/  b  o  j 

Since  i  is  uniquely  determined  by  i  =  aoboj. 

See,  Zadeh,  L.  A.  ,  Sept.  21,  1961.  See  also.  Ref.  2. 
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m-1 


[U  U,]..=  y  u  u,  =u  u, 
k=o 

^|l,  i  =  a=boj 
[0,  i  ^  aoboj 


Therefore,  by  the  definition  of 


U  U.  =  U 
— a  —  b  — ae  b 


Furthermore,  since  aob=  boa,  we  have 


U  U,  =U,  U 

— aj  b  — boa  — b — a 


Q.  E.  D. 


From  these  theorems,  we  can  derive  the  following  properties 


U  =  U. 
— m  — 0 


for  all  k;  integer  >  0 

k^'^k’  n  =  even  integer 


(2,  31) 
(2.  32) 
(2.  33) 
(2.  34) 


In  particular, 


U  ,  =  I 
— m-1  — 

Theorem  3.  U  ^  is  an  orthogonal  matrix,  i.  e.  , 


U  '  U  =  I 
—  a  — a  — 


if  and  only  if  a  ^  0  and 

GCD(a,  m)  =  1. 
m-1 


Proof. 


[U'  U  ]..  =  )  U  U 
“■—a  — a -'ll  l_i  a,  .  a 

k=0 


(2.  35) 
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From  the  definition  (2.  24)  of  U  , 


1,  k  =  a 


ol 


ap  j 


=  <  a  0 

ki  kj  0,  otherwise 

Recalling  the  theorem  of  Galois  field  that 


{aoi=aoj  mod  m}<=^{i  =  j  mod  gcDi'a/m)  ^  (2.36) 

we  have 


[U'  U  ] ..  = 
^ —  a  — a**  ij 


0,  otherwise 


Now,  i  =  j  mod  m/GCD(a,  m)  is  unique  for  i,  j  e  S,  if  and  only  if 

GCD(a,  m)  =  1.  Therefore,  U'  U  =  I  ,  if  and  only  if  a  ^  0  and 

"“a  **~a 

GCD(a,  m)  =  1.  Q.  E.  D. 

Consequently,  we  see  that  U  is  an  orthogonal  matrix  for 

•■^a 

all  ae  S  except  a  =  0,  if  m  is  a  prime.  Furthermore,  and  ^ 

are  orthogonal  matrices  for  all  integers  m  >  2. 

In  connection  with  the  determinant  of  U  ,  we  have  the  following 

•“•a 

theorem; 

Theorem  4. 


1'+  1,  if  a  0,  and  GCD(a,  m)  =  1 

liial  =1 

I  0,  if  a  =  0,  or  GCD(a,  m)  ?!  1. 

Proof.  For  a  =  0,  it  is  evident  that  |  1^  |  =  0.  If  GCD(a,  m)  =  1, 
from  the  theorem  1  there  is  exactly  one  entry  of  1  in  each  column 
and  in  each  row.  Therefore,  is  transformed  to  an  identity  ma¬ 
trix  by  a  certain  number  of  permutations  of  columns  or  rows.  The 
value  of  the  determinant  does  not  change  with  such  permutations 
except  the  sign  of  it,  which  is  positive  for  an  even  number  of  permutations 
*  See,  Zadeh,  L.A.  ^Sept.  21,  1961.  See  also.  Ref.  2. 
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and  negative  for  an  odd  number  of  permutations.  Since  |  I  [  =1,  we 
have  I  ^  1  if  a  /  0,  and  GCD(a,  m)  =  1. 

If  GCD(a,  m)  /  1,  then  by  the  theorem  1  there  exist  some 
rows  whose  entries  are  all  zeroes.  Therefore,  the  determinant 
of  it  is  zero.  Q.  E.  D. 

Example. 


m=  3, 

|Uol=o, 

lilll 

=1, 

1^2 

=  -l 

m=4. 

llill 

=1. 

=  0, 

i£3l 

=-l 

m=5, 

ISoN«. 

=1. 

1^2 

=  -l. 

1 U3I 

=  -1 

m=6. 

|u„l=o, 

iuj 

=1, 

l-y.2 

=  0, 

iiisi 

=  0 

|U4l=». 

liis 

=1 

Consequently,  we  can  conclude  that  there  exists  an  inverse 

matrix  of  1^,  if  and  only  if  a  /  0  and  GCD(a,  m)  =  1. 

So  much  for  the  general  properties  of  U  ,  we  will  consider  a 
”  —a. 

cascaded  connection  of  the  constant  multipliers  with  constants 


^1’  ^2’  ^3’ 


S. 


X  o — *(0) - 


-0  z 


Then,  from  the  theorem  2,  we  have  the  absolute  probability  of 
output  z. 


P(z  I  c,  c,  -  c,  )  =  U  U  - U  P(x) 

— '112  k'  — c, — Ct  — c, — '  ' 

12  k 


=  U 

— -C  o  c  ^ 
*^1  2 


.c 


(2.  37) 
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Next,  we  will  consider  a  modulo  m  multiplier  with  two 
random  inputs  x  and  y.  By  the  similar  fashion  to  that  which  we  used 
for  modulo  m  adder,  we  can  get 

=  UyPIxjy)  (2.38) 

Therefore,  we  have 
m-1 

P(z)  P(y)UyP(x|y) 

y=0 

or  equivalently 

m-1 

P(z)  P(x)U^P(y|x) 

x=0 

Defining  a  transition  matrix  H  such  that 

H=  [U^^(x|  0),U^P(x|l) . U^.lP(x|m-l)]  (2.40) 

we  can  rewrite  eq.  (2.  36)  in  the  form 


P(z)  =  HP(y) 


(2.  41) 


Here,  H  is  an  m  by  m  square  matrix  and  its  transpose  H' 
is  a  conditional  probability  matrix,  which  satisfies 


m-1 


j=0 


(2.  42) 


This  can  be  proved  as  follows; 
From  the  definition  of  H,  we  have 


Therefore, 


m-1 


m-1 


r  1 

(lo-! 

)  P(k|j) 

11 

1  to  J 

j=0  i=0 

From  the  property  (i)  of  U  ,  we  know  that  there  is  exactly  one  entry  of 
1  in  each  column  of  for  any  j  «  S,  hence,  the  column  sum  ofJJj  is  equal 


to  one.  Also  the  second  term  of  the  above  equation  is  one  because  P(k|  j) 
is  a  conditional  probability-  Therefore,  we  have 


m-1 


j=0 


If  X  and  y  are  statistically  independent,  then  P(x  |  y)  =  P(x)  and 


S=  . 


m-1 


x=0 


(2.43) 
(2.  44) 


Example:  Independent  case. 

m=3  m=5 


T 

^0 

^0 

^0 

^0 

0 

^1 

^3 

^2 

^4 

H  = 

0  P^ 

^2 

H= 

0 

p 

2 

^1 

^4 

^3 

0  P^ 

0 

^3 

^4 

^1 

^2 

0 

^4 

^2 

^3 

^1 

Further, 

if  P(x 

)  =  1/m  for  all  xe  S,  then  from  eq. 

(2.44) 

1 

1/m 

1/m  .  .  . 

1/m 

0 

1/m 

1/m  ... 

1/ m 

H  = 

0 

1/m 

1/m  . .  . 

1/ m 

0 

1/m 

1/m  .  . . 

1/m 
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Therefore,  we  have 


m-1 

P(y=0)+1  yp(y=i).  z  =  0 


P(2)  = 


V, 


m  i_j 
i=l 

m-l' 

i=l 


z  ^  0 


(2.  46) 


This  shows  us  that  if  P(Y=0)  is  nonzero,  the  probability  that  z=0 
is  larger  than  the  probability  that  z  takes  a  symbol  other  than  0, 
which  is  equal  for  any  symbols  z  e  S,  z^  0. 

Finally,  we  note  that  in  particular  for  the  binary  system  the 
modulo  2  multiplier  is  simply  an  on-off  switch.  If  the  two  inputs  are 
mutually  correlated  signals,  the  modulo  2  multiplier  is  a  statistically 
input-dependent  random  switch. 


III.  COMBINATORIAL  CIRCUITS 


We  will  study  here  a  few  examples  of  combinatorial  circuits  com¬ 
posed  of  the  machine  elements  studied  in  the  preceding  articles. 

A.  A  Modulo  m  Adder  and  Two  Constant  Multipliers 


z  =  aox®box=(a®b)ox  (3.1) 

Therefore,  this  circuit  is  reduced  to  a  circuit  with  a  constant  multi¬ 
plier  with  constant  (a  ©b).  Hence,  we  have 

P(z|a,b)  =Ua0b£(x)  (3.2) 
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B.  A  Delay  Element  and  a  Modulo  m  Adder 


y(t)  =  Dx(t)  =  x(t-l) 

z(t)  =  y(t)  0x(t)  =  x(t-l)  ©x(t) 


The  conditional  probability  P(x|  y)  is  now 
P  [x(t)|  y(t)]  =  P[x(t)|x(t-1)] 

Therefore,  if  the  input  process  is  a  time -invariant,  finite -state 
simple  Markov  process,  then  P(x|  y)  is  the  column  vector  composed 
of  the  y-th  column  of  the  transpose  of  the  stochastic  matrix  £  of  the 
input,  i.  e.  ,  defining 

£y=P(xly)  (3.4) 

where  the  stochastic  matrix  £  is 

£'  =  [£0’£i'£2’ •••’£m-l^ 
then,  the  absolute  probability  of  z  is 

P^(z)  =  G  P^(y)  =  G  Pt-i(x) 

where 

t£o’  . 


Using  the  property 

(3.8) 

(3.9) 


G  is  independent  of  time,  since  £  is  time-invariant, 
of  a  simple  Markov  process,  i.  e.  , 

£,+1  (“)  = 


^(x)  =  £'P^_^(x) 


we  have 


(3.  6) 

(3.7) 


-16- 


(3.10) 


^^i(z)-G£-P^_i(x) 

=  ££'G‘^Pt(x) 

=  {G'-^£G'}  '  P^(z)  (3.11) 


if  G  is  nonsingular.  This  inxplies  that  if  |  G  |  ^  0  and  x(t)  is  a  simple 
Markov  process,  then  the  output  z(t)  is  also  a  simple  Markov  process 
with  stochastic  matrix 


{G'‘^£G'  } 


Example:  m=2 


£  = 


T  = 


G  = 


^-Poi 

L  PlO 

0  r 

1  0 


i-p 


01 


Poi 


01 


1-p 


10 


1-p 


10 


10 


—  I  ■  PlO'Pol 


,-l  1 


■10 


-p 


01 


-a-Pi(^' 

(I-Poi) 


(3.12) 


^-Poi<^-Poi'^Pio)'  i-Pio(^+  PorPio> 
Poi^^-Poi^Pio^’  Pio^^’^Poi'Pio) 


G'‘^£G'  = 


^-^PoiPio  ■ 

Poi+Pid^PoiPio  ’ 


^PoiPlO 

^■PorPio'^PoiPio_ 


(3.14) 


We  will  see  in  this  example  that  if  the  input  process  is  an  independent 
process,  then 
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(3.10) 


P^^i(z)  =  G£'P^_i(x) 

=  G£'  G'^P^(x) 

=  {G'-^£G'}  '  P,(z)  (3.11) 


if  G  is  nonsingular.  This  implies  that  if  |  G|  ^  0  and  x(t)  is  a  simple 
Markov  process,  then  the  output  z(t)  is  also  a  simple  Markov  process 
with  stochastic  matrix 


{G--^£G'  } 


Example:  m=2 

^-Poi  Poi 

L  Pio  ^-Pio 

To  n 


^'PlO 

PlO 


1^1  =Pio-Poi 


g.-i=  J_ 

PlO 

-a-Pi(^ 

“  |G| 

■Poi 

(i-Poi) 

(3.12) 


l-Poi(l-Poi'''PlO^’  ^■PlO^^'*'  PqI'PiO^ 

Poi^^'Poi'^Pio)’  Pio^^'^Poi'Pio^ 


G''^£G'  = 


^■^PqiPiO  ’ 
Poi'''Pio  ^PqiPio’ 


^PqiPiO 

^■PorPio''^PoiPio_ 


(3.14) 


We  will  see  in  this  example  that  if  the  input  process  is  an  independent 
process,  then 
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Poi+  PlO=^ 

G'‘^£G'  = 


^■^PoiPlO’  ^PoiPlO 
.^■^PoiPlo’  ^PqiPiO 


(3.15) 


and,  hence,  z(t)  is  also  an  independent  process.  If  |  G|  =0,  then 
Poi=Pio  and  G  ^  does  not  exist,  and  P(x=0)=P(x=l)=l/ 2. 

1-Poi  1-p  “ 


G£' 


Poi 


'01 

Poi 


(3.16) 


and  we  have 


P,,l(z)  = 


~^-Poi 

PlO 

o 

a 

_ 1 

i“Pio. 

(3.17) 


regardless  of  P(x)  and  t. 

In  general,  if  x(t).  is  an  independent  process,  z(t)  is  also  an 
independent  process,  when  |  G|  ^  0.  The  proof  of  this  statement  is 
•given  as  follows; 

If  x(t)  is  an  independent  process,  then  p^j=Pj  and 


G': 


Pq  Pi  Pz  • 
Pm-1  Pq  Pi  ■ 
Pm- 2  Pm-1  Pq 


m-1 
^m-  2 
m-  3 


(3.18) 


Therefore,  {£  G'}  .  .={£^'}  .  is  independent  of  i. 

m-l 


.k=0 

m-l 


k=0 
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Now,  the  second  term  of  the  above  equation  is 


m-1 


m-1 


—  /(-ir'J  ICofac  G' 
n  I  i-j 


\l+j 


T=i 


j=o 


i-th  column  replaced  by  I's. 


=  |g!|/  |g1 

From  eq.  (3.18),  we  know  that 


{G'}..=p,  ,  k  =  j-  i  mod  m 

'  ij 


Therefore, 


{G'}..  = 

— n-*  ij 


1,  i  =  n 

Pj^>  k  =  j-imodm,  i,  j  €  S,  i^n 


Since  |  G^  |  does  not  change  by  an  even  number  of  permutations  of  by 
shifting  the  rows  and  columns  of  it  by  n,  i.  e.  ,  defining 


r  =  i  -  n  mod  m,  s  =  j  -  n  mod  m 


we  have 


1,  r  =  0 
k  =  j 

i  #  n,  r  ^  0 


■f  G*  1  =  i G*  T  — 

^ — n'’ij  ^ — 0'’rs  'l  Pj^’  k  =  j  -  i  =  s  -  r  mod  m 


Hence,  we  have 
m-1 


|g;|/  1g|  |G-J  /|Gl  =  Constant.  Vie  S 
j=0 


Therefore, 
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{G'‘^£G'}..  =  {G'‘^£G'}. 

Thus,  completing  the  proof.  Q.  E.  D. 

If  j  G|  =0,  then  G"^  does  not  exist  and  P^(z)  should  be  found  from 
eq.  (3.  6). 

C.  Cascaded  Circuit  of  (B) 

The  analysis  of  the  preceding  example  can  be  applied  for  the 
cascaded  circuit  shown  below. 


By  the  similar  fashion  we  have 


PAx  )  =  P  (x  ) 

—V  ji'  ^  — t-r  n' 


(3.19) 


where 

£;  stochastic  matrix  of  the  input  process 
^n)^  ^(n-l)^(n-l)'  ^q) 

G(k)  ^  ^  T  £^,^1  .  .  .  .  ,  t""- ]  (3.  21) 

G^°^  1-.  G 


By  an  iterative  method  one  can  find  the  stochastic  matrix  at  the  n-th 
stage.  Furthermore,  from  eq.  (3.  6),  we  have 


Pjx  )  =  G^'P,  (x„) 

— t  n' - ^t-n  0 


If  we  consider  a  cascaded  circuit  with  infinite  number  of 
unit  circuits,  then 
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(3.  23) 


lim  P.(x  )  =  P  (x  )  =  G  P- (x  ) 
— t  n'  00  <X)  —  —0  0' 

n — *CD 


where 


g”  ^lim  G*^ 
n— >00 


(3.24) 


Since  G'  is  a  stochastic  matrix  and  so  is  G'”^,  G°°  can  be  found  by  the  same 
way  as  we  use  to  find  the  stationary  solution  of  the  absolute  probability  Q 
of  a  Markov  process,  whose  transition  matrix  is  G' .  Therefore,  we 
have<6-8'9.11) 


G°°  =  [Q  Q  Q . Q] 

_ ) 


(3.  25) 


where  Q  is  an  eigenvector  corresponding  to  the  eigenvalue  X.=l  of  the 
character!  stic  equation 


I  =  0 

and  Q  is  uniquely  determined  from  the  condition  that 


(3.26) 


m-1 


i=0 

Example; 


m  =  2 


G'  = 


■PqI  PqI 

■PlO  PlO_ 

1 

- 1  - 

■^10 

'loPoi 

L  Pol  J 
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1 

^"PioPqI 


^-PlO 

Poi 

»— * 

1 

O 

Poi 

This  result  indicates  that  the  output  absolute  probability  of  infinitely 
cascaded  networks  of  this  type  is  independent  of  the  probability  of  the 
input  process  and 


Note  that  the  delay  introduced  through  this  circuit  is  infinity. 

State  equation  of  the  cascaded  network  with  length  n  is  written  by 


n  n-i  n-i 


and  we  can  also  express  it  in  terms  of  x^(t-i),  i. 


(t-i) 


j=0 


(3.  27) 


Accordingly,  we  can  describe  an  equivalent  circuit  with  n  delay  elements 
and  (n-1)  constant  multipliers  with  constants  c^'s. 
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D.  Delay  Element  and  Modulo  m  Multiplier 


z(t)  =  y(t)ox(t)  =  x(t-l)ox(t)  (3.  28) 

Assuming  that  x(t)  is  a  time -invariant,  simple  Markov  process,  and 
using  the  matrices  defined  in  the  preceding  article,  we  have 

P^(z)  =  H^_^(x)  (3.29) 

where  H  is,  for  this  particular  case, 


[Uq£q,  U,£^,U2£2. 


U  -1  P  -1  , 
— m-1  — m-i 


(3.  30) 


£j^  is  defined  by  eq.  (3.  4)  from  the  stochastic  matrix  of  the  input  process. 
Similar  to  the  preceding  example,  we  have 


=  Ji£'  Zt-iW  (3.  31) 

P^^^(z)  =  {H'  £  H-}  -  P^(z)  (3.  32) 

This  implies  that  if  the  input  process  is  a  simple  Markov  process  and  |  H|  i^O, 
then  the  output  process  is  also  a  simple  Markov  process  with  stochastic 
matrix 

{H'“^£H'}  (3.33) 
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Ul=i 


H= 


1 

0 


P  n 
10  • 


^-PioJ 


H  I  =  1  -  p 


10 


*  ^  —  1 

—  1-p 


10 


1-p, 


10 


H£'  = 


LPoi^^'Pio^ 


-p 


10 


l-Poi(l-Pio)  ^Plo'PlO 


(1-Pio) 


2  ^ 
) 

2 


(3.  34) 


H'  £H'  = 


I'Poi^^'PlO^  Pin^^’Pin) 


10'  no' 


L^10^^'''P01^  ^‘PlO^^'^Poib 


(3.35) 


From  this  we  can  see  that  if  the  input  process  is  an  independent  process, 
then  Pqj^+Pj^q  “  ^  ^.nd  the  output  process  is  also  an  independent  process. 


E.  Cascaded  Circuit  of  (D) 

X, 


D 


_n^ 


D  ^  o 


If  we  define  the  state  vectors  as  in  the  above  figure,  then 
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and  finally  we  have 


x^(t)  =  n  {x_(t-k)} 


k^O 

where 

Co 

X  =  X  C  Xo  XO  , 


ox 


(3.56) 


n  X,  =  X.  o  X,  O  X-O. 
,  K  0  1  2 

k=0 


ox 


n 


^  \k 

The  results  are  quite  similar  to  the  example  in  (C). 


where 

(0) 

£  =  £ 


H(k)_  rTj  (k)  „  (k) 

£1  -  ’  i^l^l  ’ 

=  H 


Then  the  input -output  relation  is  given  by 


H 


(n-1)' 


(3.57) 


Zt<"n)  = 

Also  the  limiting  absolute  probability  P^(x^)  is 


P  (x  )  =  Q 

- CO  oo  — 


where 


=  [Q  Q  Q 


m 


Q] 


(3.  58) 


(3.59) 
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and  Q  is  the  stationary  solution  of  the  Markov  process,  which  satisfies 


Example:  m=2 


(I-H)  Q  =Q 


H  = 


PlO 


LO  I-p^qJ 


(3.  60) 


Q  = 


=  ) 
—  (» 


1  r 
0  0 


In  general  we  have 


I  1  1 
0  0  0 
0  0  0 

0  0  0 


P  (x  )  = 
— oo  oo 


1 

0 

0 


1 

0 

0 

0 


0 


(3.61) 


(3.62) 


This  implies  that  the  output  symbol  is  0  with  probability  1  as  n  approaches 
infinity.  It  is  quite  natural  because  "0"  is  an.  absorbing  state  through  this 
network,  and  by  an  infinitely  large  number  of  trials  through  these  network 
elements,  the  symbol  at  the  final  stage  is  absorbed  in  "0." 
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IV.  CONCLUSIONS 


We  have  studied  the  transformation  matrices  T  and  U  and  their 

"3, 

properties.  Further,  defining  the  transition  matrices  G  and  H  for  a 
modulo  m  adder  and  for  a  modulo  m  multiplier  derived  from  ^  and  U^, 
the  output  probabilities  are  expressed  simply  by  the  product  of  these 
matrices  and  the  input  probability  matrix. 

Also  we  have  studied  a  few  examples  of  the  combinatorial  circuit 
using  these  modular  circuit  elements.  Circuits  with  delay  elements  are 
particularly  of  our  interest.  We  have  shown,  in  a  few  examples,  that 
if  the  input  process  is  a  time -invariant  simple  Markov  process,  then  the 
output  process  is  also  a  simple  Markov  process  whose  stochastic  ma¬ 
trix  is  iteratively  found  from  the  input  stochastic  matrix. 

Combinatorial  circuits  with  feedback  are  not  treated  yet  in  this 
paper.  The  same  approach  would  be  applied  for  a  circuit  with  feed¬ 
back,  which  is  now  being  studied.  If  we  obtain  some  expressions  for 
them,  it  would  be  possible  to  generalize  this  analysis  for  a  general 
modular  system. 

In  a  digital  communication  system,  or  in  a  digital  data  processing 
system,  one  may  assume  a  digital  computer  or  a  sequential  circuit 
at  the  terminal  machines  of  it.  Its  input  will  be  a  stochastic  process. 
Then,  what  would  be  an  output  process?  This  question  is  not  wholly 
answered  yet,  because  the  circuit  with  feedback  is  not  analyzed  yet. 

It  would  be  very  interesting  to  analyze  the  probabilistic  properties 
of  such  systems  in  order  to  design  a  desired  circuit  or  to  optimize 
a  system  in  a  probabilistic  sense. 
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APPENDIX 


Examples  of  U^: 


m  =  2 

‘1 

0  0 


m  =  4 


1  1 
0  0 
0  0 
-0  0 


1  1 
0  0 
0  0 
0  0 


^1= 


1  0 
0  1 
0  0 
0  0 


0  0 
0  0 
1  0 
0  1 


0 

0 

1 

0 


1  0 
0  0 
0  1 
0  0 


0 

0 

0 

1 


0  0 
0  1 
1  0 
0  0 


m  =  3 

~1  1  1~ 
0  0  0 
_0  0  0_ 

"1  0  0~ 
0  1  0 
-0  0  1_ 


m  =  5 


U2= 


1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

_0 

0 

0 

0 

0_ 

~1 

0 

0 

0 

0~ 

0 

1 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

1 

0 

_0 

0 

0 

0 

1 

1. 

0 

0 

0 

0“ 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

1 

_0 

0 

1 

0 

0_ 

"I 

0 

0 

0 

o" 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

0 

1 

0 

0 

0 

_0 

0 

0 

1 

0_ 

'l 

0 

0 

0 

o" 

0 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

0  0 
0  1 
1  o_ 
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m  =  6 


1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

J) 

1 

"o 

0 

0 

0 

0 

0 

0 

0 

1 

1  0  0 

1 

'0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

IL 

_Q_ 

0 

0 

6” 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

0 

0_ 

1 

0 

1 

o” 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

1 

0 

0 

1 

0 

0 

Q_ 

JL 

1 

0 

r 
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